Towards Multi-tenant GPGPU: Event-driven Programming Model for System-wide Scheduling on Shared GPUs

Graphics processing units (GPUs) are attractive to the generalpurpose computing (GPGPU) beyond the graphics purpose. Sharing GPUs among such GPGPU applications is a key requirement especially for cloud platforms whose resources are utilized by various cloud users. However, consolidating recent GPU applications, referred to as GPU eaters, on a GPU poses a new challenge. Such advanced applications are designed based on the assumption that only one GPU application runs on a GPU at a time. In this paper, we present GLoop, a GPGPU framework that allows multiple GPU eaters to share a GPU. GLoop offers an event-driven programming model that generates scheduling points in GPU kernels. It achieves resource isolation among GPU applications and schedules applications by suspending/resuming GPU kernels only if necessary to reduce the number of kernel launches. The preliminary experiments demonstrate that GLoop model is applicable to a GPU eater. The results also show that GLoop enables two GPGPU applications to run concurrently on a shared GPU by interleaving GPU kernel execution.